Skip to content

Add Flame engine to the registry#489

Merged
auto-submit[bot] merged 5 commits into
flutter:mainfrom
spydon:add-flame-tests
Jul 6, 2026
Merged

Add Flame engine to the registry#489
auto-submit[bot] merged 5 commits into
flutter:mainfrom
spydon:add-flame-tests

Conversation

@spydon

@spydon spydon commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Adds the Flame engine to the customer test registry so that its tests run as a presubmit against flutter/flutter framework changes.

How it works

Flame is a monorepo managed as a pub workspace, so the update step (flutter pub get at the repo root) resolves every member package in one go. The test command runs scripts/customer_testing.dart in the Flame repository, which works identically on every platform (no separate .sh/.bat needed):

  • flutter analyze --no-fatal-infos runs over the whole workspace on every platform.
  • The full per-package flutter test suites (27 packages) run on Linux only, because Flame's golden and rendering tests are platform-sensitive. This mirrors how flutter/packages handles its own registry entry.

Notes for reviewers

  • The pinned revision points at the branch commit of test: Add customer_testing script for the flutter/tests registry flame-engine/flame#3941, which adds scripts/customer_testing.dart. Once that PR merges I'll update the SHA here to the resulting main commit before this lands.
  • Flame's full suite (~300 test files) is larger than a few-seconds suite. Analyze is fast on all platforms; the Linux test run is where the bulk of the time goes. Happy to trim the package set if the total runtime is a concern.
  • An earlier attempt to register Flame lived on the old master branch and predates pub workspaces (it relied on melos bootstrap); this replaces that approach.

Locally verified that root flutter pub get resolves the workspace, the analyze pass is clean over the whole tree, and per-package flutter test works.

Runs Flame's tests (https://github.com/flame-engine/flame) as a presubmit
against flutter/flutter. Flame is a pub workspace; scripts/customer_testing.dart
runs flutter analyze over the whole workspace on every platform and the
per-package test suites on Linux only (goldens are platform-sensitive).
@spydon spydon force-pushed the add-flame-tests branch from f4427ce to 4bcacf5 Compare July 3, 2026 10:27
spydon added a commit to flame-engine/flame that referenced this pull request Jul 3, 2026
## Description

Adds `scripts/customer_testing.dart` so that Flame can be registered in
the [flutter/tests
registry](https://github.com/flutter/tests/tree/main/registry).

The registry runs downstream packages' tests as a presubmit against
`flutter/flutter` framework changes, so that breaking changes to the
framework that would affect Flame are caught early (and Flame's code can
be auto-migrated via `dart fix`).

A companion PR is open against `flutter/tests`: flutter/tests#489. It
adds a `registry/flame.test` entry pinned to the commit of this PR that
invokes the script via `dart run scripts/customer_testing.dart`.

## Why a single Dart script

The registry guarantees `dart` on the path, and a single Dart file is
inherently cross-platform, so there's no need for the separate
`.sh`/`.bat` pair the registry template otherwise expects. It also
matches the existing `scripts/test_formatter.dart`.

## Behavior

- Flame is a pub workspace, so the registry's `update` step (`flutter
pub get` at the repo root) resolves every member package.
- `flutter analyze --no-fatal-infos` runs over the whole workspace on
**every platform**.
- The full per-package `flutter test` suites run on **Linux only**,
since golden and rendering tests are platform-sensitive. This mirrors
how `flutter/packages` handles its own registry entry.

Locally verified: root `flutter pub get` resolves the workspace, the
script's analyze pass is clean over the whole tree, and per-package
`flutter test` works.
Comment thread registry/flame.test Outdated
spydon added a commit to supabase/supabase-flutter that referenced this pull request Jul 6, 2026
## What

Adds `scripts/customer_testing.dart`, the entry point invoked by the
[flutter/tests](https://github.com/flutter/tests) registry to run this
repository's tests as a presubmit against `flutter/flutter` framework
changes.

This is the supabase-flutter counterpart of what Flame did in
[flutter/tests#489](flutter/tests#489). A
companion PR to `flutter/tests` adds the
`registry/supabase_flutter.test` entry that clones this repo at a pinned
commit and runs this script.

## What the script does

1. `flutter analyze --no-fatal-infos` over the whole pub workspace
(covers every member package).
2. `flutter test` in `packages/supabase_flutter`.

Only `supabase_flutter` is exercised because it is hermetic (mock/stub
based). The other packages' tests require a live Supabase backend and
are therefore not eligible for the registry, which requires hermetic
tests (no network, no spawned services).

## Why

So that Flutter framework regressions affecting `supabase_flutter` are
caught upstream, before they reach a stable Flutter release.

## Notes

- The registry only honors changes here once the pinned commit in the
flutter/tests entry is updated (revision bumps are periodic PRs to
flutter/tests).
- Verified locally: `dart run scripts/customer_testing.dart` passes
(analyze clean, 57 tests pass).


Part of SDK-1216

@Piinks Piinks left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thank you for contributing tests!

@auto-submit

auto-submit Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

autosubmit label was removed for flutter/tests/489, because This PR has not met approval requirements for merging. The PR author is not a member of flutter-hackers and needs 1 more review(s) in order to merge this PR.

  • Merge guidelines: A PR needs at least one approved review if the author is already part of flutter-hackers or two member reviews if the author is not a member of flutter-hackers before re-applying the autosubmit label. Reviewers: If you left a comment approving, please use the "approve" review action instead.

@chunhtai chunhtai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@auto-submit auto-submit Bot merged commit 722231d into flutter:main Jul 6, 2026
18 checks passed
auto-submit Bot pushed a commit that referenced this pull request Jul 6, 2026
Adds a registry entry for [supabase-flutter](https://github.com/supabase/supabase-flutter), the official Supabase client library for Flutter/Dart, so that framework changes are checked against its tests as a presubmit.

Follows the same pattern as the Flame entry (#489): the entry clones supabase-flutter at a pinned commit and runs `scripts/customer_testing.dart`, which:

1. Runs `flutter analyze --no-fatal-infos` over the whole pub workspace (covers every member package).
2. Runs `flutter test` in `packages/supabase_flutter`.

Only `supabase_flutter` is exercised because it is hermetic (mock/stub based). The repo's other packages have tests that require a live Supabase backend, so they are not included here.

The `scripts/customer_testing.dart` script is added in supabase/supabase-flutter#1528.

contact: lukas.klingsbo@supabase.io
flutter-zl pushed a commit to flutter-zl/flutter that referenced this pull request Jul 7, 2026
This bumps the customer testing commit to pick up tests from 
- flutter/tests#490
- flutter/tests#489

FYI @spydon :)

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [AI contribution guidelines] and understand my
responsibilities, or I am not using AI tools.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [ ] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

If this change needs to override an active code freeze, provide a
comment explaining why. The code freeze workflow can be overridden by
code reviewers. See pinned issues for any active code freezes with
guidance.

**Note**: The Flutter team is currently trialing the use of [Gemini Code
Assist for
GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code).
Comments from the `gemini-code-assist` bot should not be taken as
authoritative feedback from the Flutter team. If you find its comments
useful you can update your code accordingly, but if you are unsure or
disagree with the feedback, please feel free to wait for a Flutter team
member's review for guidance on which automated comments should be
addressed.

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[AI contribution guidelines]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants